Automatic Acquisition of Linguistic Knowledge: From Sinica Corpus to Gigaword Corpus

The raison d’etre for a corpus, as it was first conceived by Francis and Kucera in 1963, was to provide a body of linguistic facts from which linguistic knowledge could be generalized, [1]. The methods of acquisition have evolved as corpus size and technology have advanced in the past 40 years. Originally corpus-based concordances assisted linguists to form generalizations. This was what Fillmore [2] characterized as a ‘computer-aided armchair linguist’. Today, direct, automatic acquisition of linguistic knowledge from a corpus is becoming a reality. Two trends that are critical to the automatic acquisition of linguistic knowledge from a corpus are the increase in corpus size and the development of technology to extract linguistic relations. The release of the Chinese Gigaword corpus [3] by LDC in 2004 set the stage for a billion (1,000,000,000) word corpora; while the development of Sketch Engine by Adam Kilgarriff and colleagues [4] in the same year provided tools for the automatic acquisition of linguistic knowledge. Unlike the balanced corpus tradition established by the Brown Corpus and adopted by the Sinica Corpus (1995, the first annotated Chinese corpus) [5], the Gigaword Corpus has a uniform data source. It consists of two sub-corpora: one from the Central News Agency in Taiwan and the other from the Xinhua News Agency in PRC. In other words, the Gigaword corpus is a gargantuan news corpus representing the two major variants of Mandarin Chinese. The sheer size of the Gigaword Corpus poses both a challenge and an opportunity. First, the challenge lies in how to achieve a high quality corpus annotation with a minimal of human intervention. The current standard procedure of corpus annotation, especially POS tagging, is automatic tagging with human post-editing. It is impractical to adopt the standard post-editing procedure for the Gigaword corpus because of the scale of the undertaking. Instead, we apply the Academia Sinica tagging system developed for the construction of the Sinica Corpus, with the statistical model trained on its complete 5 million word corpus. In addition, lexicon adaptation, unknown word detection and feedback modules are implemented. The result is a truly automatic, highly efficient annotation program that creates a fully tagged Gigaword Corpus. The Gigaword corpus also provides the possibility of automatic extraction of grammatical relations. Automatic assignment of syntactic structure is a difficult NLP task. A precise structural assignment for a specific sentence or construction at the level that a linguist would desire is still impossible. However, such parochial errors become negligible when grammatical relations are extracted based on significant patterns of a large number of examples. This is the design criteria of the Sketch Engine (Kilgarriff et al. [4]) and the same criteria has been applied to the annotated Gigaword Corpus in order to construct the Chinese Sketch Engine. Our early experiments showed that the grammatical information extracted is generally reliable, although the interpretation of the acquired information must still be carried out with the aid of linguistic expertise. In conclusion, recent developments in corpus linguistics clearly point toward billion-word size, fully automatic annotation, and automatic acquisition of linguistic knowledge. These developments will shape the construction of future corpora.